Papers with automated optimization
Steering Language Models in Multi-Token Generation: A Case Study on Tense and Aspect (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has focused largely on binary grammatical contrasts, but how do they encode their syntactic knowledge internally? |
| Approach: | They propose to use a multidimensional hierarchical grammar phenomenon to identify distinct, orthogonal directions in residual space to demonstrate causal control over both grammatical features. |
| Outcome: | The proposed model can encode tense and aspect in human-like ways, but effective steering during generation is sensitive to multiple factors and requires manual tuning or automated optimization. |